5 Global Optimization

12 Our view is that no general purpose algorithm should be expected to eeciently nd global mini-mizers of nonconvex continuous optimization problems. We submit that eecient global optimization is generally impossible|see, for example, Stephens and Baritompa (1998) and Anonymous (1972). But this does not mean that one should not endeavor to devise algorithms for speciic global optimization problems. In doing so, one may do well to exploit the ideas that underlie existing methods. The ideas that underlie simulated annealing are mathematically rigorous and may ultimately serve even the most vocal critics. Acknowledgments This exposition of simulated annealing is based on lectures delivered at Rice University in December 1996 and January 1997. I thank the members of Richard Tapia's research group for their interest and Amr El-Bakry for inviting me to reprise my presentation. I also thank Rex Kincaid for reading a draft manuscript and making helpful suggestions.tions of state calculations by fast computing machines. 11 The basic idea of simulated annealing is to observe the asymptotic behavior of X t , using as a continuation parameter. We are thus led to study the behavior of nonstationary Markov chains in which the transition probabilities vary with. Further theory, in which the critical issue is the nature of (t), is then required to establish convergence to x. This theory is beyond the scope of our exposition, but we hope that we have persuaded the reader that such theories can be based on mathematics rather than on metaphor. 7 Discussion The physical interpretation of simulated annealing derives from statistical mechanics. Let f(x) = U(x) denote the potential energy of a connguration x. Let = 1 kT ; where T denotes temperature and k is the Boltzmann constant. Then is the Boltzmann probability density function for the state of the connguration at temperature T. The global minimum energy connguration is obtained by letting T ! 0, i.e. by cooling. Thus, the (t) that arises in simulated annealing is the \cooling schedule". Since the widely cited article by Kirkpatrick, Gelatt, and Vecchi (1983), the physical interpretation of simulated annealing has been a remarkably eeective advertisement for its use. We submit that a healthier perspective is that the convergence theory for simulated annealing illuminates the natural phenomenon of physical annealing. Consider, for example, the tremendously important issue of how rapidly simulated annealing converges to the global minimizer. The Metropolis transition probabilities are p ij = min …